Method and apparatus for generation or editing of layer delineations

ABSTRACT

Methods are disclosed for the generation and editing of layer delineations within three-dimensional tomography scans. Cross sections of a subject are generated and presented to an operator, who has the ability to edit layer delineations within the cross section, or determine parameters used to generate new cross sections. By guiding an operator through a set of displayed cross sections, the methods can allow for a more rapid, efficient, and error-free segmentation of the subject. The cross sections can be nonplanar in shape or planar and non-axis-aligned. The cross sections can be restricted to exclude one or more user-defined regions of the subject, or to include only one or more user-defined regions of the subject. The cross sections can be localized to a point-of-interest. Iterative implementations of the methods can be used to arrive at a segmentation deemed satisfactory by the user.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/492,165, filed on Apr. 20, 2017, which claims the benefit of andpriority to U.S. Provisional Application No. 62/325,411, filed on Apr.20, 2016, each of which is hereby incorporated by reference in itsentirety.

FIELD OF THE INVENTION

The present invention generally related to methods and systems for thegeneration and editing of layer delineations within three-dimensionaltomography scans. In preferred embodiments, cross sections of a subjectare generated and presented to an operator, who has the ability to editlayer delineations within the cross section, or determine parametersused to generate new cross sections. The cross sections can be planarnonplanar in shape or axis or non-axis-aligned. The cross sections canbe restricted to exclude one or more user-defined regions of thesubject, or to include only one or more user-defined regions of thesubject. The cross sections can be localized to a point-of-interest.Iterative implementations of the methods can be used to arrive at adelineation deemed satisfactory by the user.

BACKGROUND

Optical coherence tomography (OCT) is a well-established medical imagingtechnique used to create three-dimensional (3D) images of, for example,the retina. Analysis of images recorded using OCT can delineate thelayers of the retina through a procedure referred to as retinal layersegmentation. Such technologies are used clinically to determinevariations in the relative thickness of certain retinal layers, as thesethicknesses have been shown to relate to various pathologies. In manycases, a choice of treatment can depend in part on parameters such asretinal layer thicknesses. As the eye is an extension of the centralnervous system, technologies associated with the imaging and delineationof retinal layers have medical uses beyond those limited to thedetection and management of ocular pathologies.

Advanced stage ocular diseases tend to severely disrupt the normalarchitecture of the retina, which can increase the likelihood of failurefor automated algorithms used for retinal layers segmentation. In suchcases, however, the retinal structure and its response to treatmentstill needs to be accurately assessed. This can require input from anoperator, such as a researcher or clinician, to edit pre-existing layerdelineations, or to generate entirely new layer delineations. Editing oflayers is done based on repeatedly working with two-dimensional (2D)images extracted from the 3D volumes. The efficiency of methods forgenerating and editing these layer delineations can significantly impacttheir utility and robustness.

BRIEF SUMMARY

In general, provided herein are methods for determining biological layerstructure in a three-dimensional (3D) tomography scan. One providedmethod comprises rendering a first two-dimensional (2D) image using acomputer. The first 2D image is produced from a 3D tomographic modelusing the computer. The 3D tomographic model represents a subject havingan x-axis, a y-axis, and a z-axis. The first 2D image represents a firstcross section of the subject. The first cross section is theintersection of the subject and a first surface. The first surface isparallel to the z-axis. The method further comprises displaying thefirst 2D image using the computer. The method further comprisesrecording operator input of a first delineation of one or more layerswithin the displayed first 2D image using the computer. The methodfurther comprises relating the first delineation with a first layer ofthe one or more layers using the computer. The method further comprisesrendering a second 2D image produced from the 3D model using thecomputer. The second 2D image represents a second cross-section of thesubject. The second cross section is the intersection of the subject anda second surface. The second surface is either (i) a nonplanar surface,wherein the nonplanar surface is parallel to the z-axis; or (ii) aplanar surface, wherein the planar surface is parallel to the z-axis,and wherein the planar surface is not parallel to any plane comprising(a) the z-axis and (b) either the x-axis or the y-axis. The methodfurther comprises displaying the second 2D image using the computer. Themethod further comprises recording operator input of a seconddelineation of one or more layers within the displayed second 2D imageusing the computer. The method further comprises associating the seconddelineation with the first layer using the computer. The method furthercomprises generating layer data for the 3D tomographic model byinterpolating the first and second delineations using the computer. Themethod further comprises outputting layer data for the 3D tomographicmodel using the computer.

In some embodiments, the method further comprises rendering a third 2Dimage produced from the 3D tomographic model using the computer. Thethird 2D image represents a third cross section of the subject. Themethod further comprises computing a third delineation of one or morelayers within the displayed third 2D image using the computer. Themethod further comprises displaying the third 2D image and the thirddelineation of layers using the computer. The method further comprisesaccepting operator input editing the third delineation of layers usingthe computer, thereby generating an edited third delineation of layers.The method further comprises associating the edited third delineationwith the first layer using the computer. The method further comprisesgenerating layer data for the 3D tomographic model by interpolating thefirst and second delineations and the edited third delineation using thecomputer.

In some embodiments, the second surface is a planar surface. The planarsurface is parallel to the z-axis of the subject. The planar surface isnot parallel to any plane comprising (a) the z-axis of the subject and(b) either the x-axis or the y-axis of the subject.

In some embodiments, the second surface is a nonplanar surface. Thenonplanar surface consists of each line parallel to the z-axis of thesubject and passing through a first path in an x-y plane of the subject.The x-y plane of the subject comprises the x-axis and the y-axis of thesubject. In some embodiments, the first path in the x-y plane of thesubject is a circle.

In some embodiments, the generating of layer data is through the use ofa scattered point interpolation algorithm.

In some embodiments, the 3D tomographic model is produced from two ormore tomographic images. In some embodiments, the tomographic images areoptical coherence tomography images. In some embodiments, the subject isa retina. In some embodiments, the delineation of one or more layers isa retinal layer segmentation.

In some embodiments, the method further comprises rendering a third 2Dimage produced from the 3D tomographic model using the computer. Thethird 2D image represents a third cross section of the subject. Thethird cross section is the intersection of the subject and a thirdsurface. The third surface consists of each line parallel to the z-axisof the subject and passing through a second path in the x-y plane of thesubject. The method further comprises displaying the third 2D imageusing the computer. The method further comprises recording operatorinput of a third delineation of one or more layers within the displayedthird 2D image using the computer. The method further comprisesassociating the third delineation with the first layer using thecomputer. The method further comprises generating layer data for the 3Dtomographic model by interpolating the first, second, and thirddelineations using the computer.

In some embodiments, the method further comprises accepting operatorinput defining a second path in the x-y plane of the subject using thecomputer. The method further comprises rendering a third 2D imageproduced from the 3D tomographic model using the computer. The third 2Dimage represents a third cross section of the subject. The third crosssection is the intersection of the subject and a third surface. Thethird surface consists of each line parallel to the z-axis of thesubject and passing through the second path in the x-y plane of thesubject. The method further comprises displaying the third 2D imageusing the computer. The method further comprises recording operatorinput of a third delineation of one or more layers within the displayedthird 2D image using the computer. The method further comprisesassociating the third delineation with the first layer using thecomputer. The method further comprises generating layer data for the 3Dtomographic model by interpolating the first, second and thirddelineations using the computer.

In some embodiments, the method further comprises determining a seriesof confidence values using the computer. Each confidence value isassigned to a predetermined point in the x-y plane of the subject. Themethod further comprises defining the second path in the x-y plane ofthe subject using the computer. The second path is selected to comprisepoints in the x-y plane having a mean or minimum confidence value belowa target value.

In some embodiments, the confidence value assigned to each predeterminedpoint is a function of the minimum distance between the predeterminedpoint and the first or second cross section. In some embodiments, theconfidence value assigned to each predetermined point is a function ofthe output of an edge detection algorithm comparing data from the 3Dtomographic model at that predetermined point with data from the 3Dtomographic model at an adjacent predetermined point.

In some embodiments, the method further comprises collecting operatorinput labeling zero, one, or more regions in the x-y plane of thesubject as desired using the computer. The method further comprisescollecting operator input using the computer labeling zero, one, or moreregions in the x-y plane of the subject as undesired using the computer.The second path is selected to consist of points within any regionslabeled as desired, or the second path is selected to consist of pointsoutside any regions labeled as undesired.

In some embodiments, the method further comprises accepting operatorinput defining a point-of-interest in the x-y plane of the subject usingthe computer. The method further comprises defining a second path in thex-y plane of the subject using the computer. The second path is selectedto comprise and center about the point-of-interest. The method furthercomprises rendering a third 2D image produced from the 3D tomographicmodel using the computer. The third 2D image represents a third crosssection of the subject. The third cross section is the intersection ofthe subject and a third surface. The third surface consists of each lineparallel to the z-axis of the subject and passing through the secondpath in the x-y plane of the subject. The method further comprisesdisplaying the third 2D image using the computer. The method furthercomprises recording operator input of a third delineation of one or morelayers within the displayed third 2D image using the computer. Themethod further comprises associating the third delineation with thefirst layer using the computer. The method further comprises generatinglayer data for the 3D tomographic model by interpolating the first,second, and third delineations using the computer.

In some embodiments, the second path traces a spiral shape. In someembodiments, the second path traces a multi-arm star shape. In stillfurther embodiments, the second path traces could be one or more othergeometric patterns.

Also provided is a method of determining biological layer structure in a3D tomography scan comprising rendering a 2D image produced from a 3Dtomographic model using a computer. The 3D tomographic model representsa subject. The 2D image represents a cross section of the subject. Thecross section is the intersection of the subject and a nonplanarsurface. The method further comprises displaying the 2D image using thecomputer. The method further comprises recording operator input of adelineation of one or more layers within the displayed 2D image usingthe computer. The computer further comprises relating the delineationwith a first layer of the one or more layers using the computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a rendered OCT image and retinal layer delineation inaccordance with an embodiment.

FIG. 2 illustrates cross sections (i.e., surfaces) of a subject inaccordance with an embodiment.

FIG. 3 illustrates an iterative workflow in accordance with anembodiment.

FIG. 4 illustrates a zone-based workflow in accordance with anembodiment.

FIG. 5 illustrates an exemplary embodiment of a method for generation orediting of layer delineations, in accordance with an embodiment of thepresent invention.

FIG. 6A-6C illustrates an exemplary embodiment of a method forgeneration or editing of layer delineations, in accordance with anembodiment of the present invention.

FIG. 7 illustrates a schematic overview of a computing device, inaccordance with an embodiment of the present invention;

FIG. 8 illustrates a schematic overview of an embodiment of a system forgeneration or editing of layer delineations; and

FIG. 9 illustrates a schematic overview of an embodiment of a system forgeneration or editing of layer delineations.

DETAILED DESCRIPTION

A method of determining biological layer structure in athree-dimensional (3D) tomography scan is described. The method can beused in a workflow that increases the speed and efficiency of editingpre-existing layer delineations or generating new layer delineations.

For the purposes of this detailed description, editing may broadlydefined to include activities relating to the changing and creating oflayer boundaries as rendered along with the image data in accordancewith embodiments of the present invention. Editing can further mean theadjustment of an existing layer, deletion, addition, cutting, pasting,cropping, merging or combining of layers, moving of layers, or anycombination thereof. Editing may be generally relative to the image dataand positions created via the planar- or non-planar and axis- ornon-axis-aligned surfaces. One of ordinary skill in the art wouldappreciate that there are numerous types of editing that could beaccomplished through embodiments of the present invention, andembodiments of the present invention are contemplated for use with anyappropriate forms of editing.

Severe ocular diseases can result in disrupted retinas, yet clinical endpoints require accurate measurements of structure, which in some casesrequires manual intervention. Furthermore, due to the high average ageof patients suffering from neurodegenerative diseases, structuralanalyses of atrophying neuronal layers is often complicated by otherocular diseases common in this aging population. In many such cases,only expert interpretation can tease out accurate parameters andbiomarkers from the data. These manual review and edits resulting in newlayer delineations, however, are currently extremely laborious (˜2 hoursper scan).

FIG. 1 shows an optical coherence tomography (OCT) image of the humanretina. On the right-hand side of FIG. 1 is shown delineation of layersof retina derived using a fully automated segmentation algorithm(retinal layer segmentation). Previously disclosed methods forcorrecting layer segmentations such as those shown in FIG. 1 involveediting in a “slice-by-slice basis” along one of the x-axis or y-axis ofthe biological sample. If a large portion of the delineated layer isincorrect, it can take a very long time to edit the entire layer. Thisslice-by-slice editing can suffer from being time-consuming andlabor-intensive, requiring an average of 2 hours per scan to complete.

The workflows disclosed herein can guide users through a set ofcanonical segmentations via a “wizard”-like interface. Thesesegmentations can require many fewer manual edits and help the userarrive at a layer delineation much more quickly.

FIG. 2 illustrates cross sections, or slices, of a subject in accordancewith an embodiment. Shown in FIG. 2 is a biological sample having anx-axis, a y-axis, and a z-axis. The z-axis is typically defined to bethe direction from the sample to the tomographic imager. In this way,the z-direction typically relates to depth within the sample, with anx-y plane being perpendicular to this z-axis. Also shown are three crosssections of the subject (10, 11, 12). The first 10 and second 11 crosssections are formed from intersections of the sample with surfaces thatare planar and non-axis-aligned. The third cross 12 section is formedfrom the intersection of the sample with a surface that is nonplanar.Also shown is a “top-down” view of the cross sections, showing the pathsthat the cross sections trace in the x-y plane of the sample.

In some embodiments, the user is prompted to manually segment a specificset of predefined slices of a 3D tomographic model representing thesubject. The slices can be chosen to optimize coverage with minimal userinteraction. This coverage can be optimized by minimizing the maximum,mean, or total distance from an arbitrary location to a predefinedslice. This coverage can be optimized by minimizing the maximum, mean,or total distance from all “low confidence” locations. Low-confidencelocations can be defined as locations having layer segmentations deemedless likely to be correct. This coverage can be optimized by usingpre-defined slices that partition the sample into separate zones. Thezones can be optimized to minimize the maximum or mean area.

FIG. 3 illustrates an iterative workflow in accordance with anembodiment. In step (i) of the workflow of FIG. 3 , software generates anew layer based an operator's delineations of the predefined slices. Instep (ii) of the workflow of FIG. 3 , the operator draws a continuouscurve (depicted here passing through a first cross section at 20, asecond cross section at 21, and back through the second cross section at22) through an area or areas that need more accurate segmentations.Alternatively, the algorithm itself can suggest a curve or continuousset of x,y positions based on a number of criteria. The curve can besuggested to cover a region with the highest likelihood of error. Thecurve can be suggested to minimize the maximum, mean, or total distancefrom an arbitrary point to an edited slice. The curve can be suggestedto minimize the maximum, mean, or total distance from any low-confidencepoint to an edited slice. In step (iii) of the workflow of FIG. 3 , aslice corresponding to this curve is generated through the volume andthe current layer delineation is displayed (note that in this exemplaryembodiment, the slice generated unwraps the operator's curve, therebypassing again through the smaller circular cross section at 20 and thelarger circular cross section at 21 and at 22). The user can then beprompted to either edit the current layer delineation or generate a newone. After the user is finished editing the layer delineation of theslice, a new segmentation is generated in step (iv) of the workflow ofFIG. 3 . The workflow can iterate until a user is satisfied with thefull segmentation.

Areas of “low confidence” can be derived from a likelihood map thatwould characterize the correctness or consistency of the remainingsegmentation locations. The likelihood map can have large values atlocations where there is a good chance the current layers are incorrect.The likelihood map can be generated with the formula:L(x,y)=N(x,y)/C(x,y)where N(x,y) represents the proximity of location x,y to an existinguser edit (i.e., the closer a point is to a user edit, the more likelyis s to be correct). C(x,y) represents the confidence in the automatedsegmentation value. A high confidence can indicate a low likelihood forediting. C(x,y) can be calculated in a number of ways. C values candepend on similarity to neighboring manually segmented areas. C valuescan depend on overall edge strength.

Additionally, and at any stage, the user can be able to explicitlydesignate “don't care” regions that will be essentially masked from allconsideration and influence from the remaining, implicitly defined “docare” regions. This can be necessary in cases involving regions ofextremely low signal. These can result, for example, from opacities inthe eye preventing any light reaching certain areas of the retina. Insuch cases an editing wizard can otherwise continually return to theseareas that are not of interest. Also, it is preferred that such poordata not influence data related to neighboring areas. This designationof “don't care” regions can provide a mechanism to ensure that this isnot the case.

In some embodiments, by default all the data is initially designated a“do care” region. Once editing is completed, the user can still segmentthe “don't care” regions outlined. The results from neighboring areascan be used to complete the segmentation (either by simple interpolationor by explicit segmentation).

FIG. 4 illustrates a zone-based workflow in accordance with anembodiment. In step (i) of the workflow of FIG. 4 , the softwaregenerates a new segmentation based on a user's delineations ofpre-defined slices. In step (ii) of the workflow of FIG. 4 , the userselects a zone that needs editing, where each zone is an area bounded bypre-defined slices. A slice corresponding to a curve that covers thezone is generated through the volume, and the current layer delineationfor that slice is displayed in step (iii) of the workflow of FIG. 4 .The user is then prompted to either edit this current layer delineationor to generate a new one. When the user is finished editing the layerdelineation of the current slice, a new segmentation is generated instep (iv) of FIG. 4 . The user can iterate over the zones untilsatisfied with the full segmentation.

The provided methods can also be used for the editing of layerdelineations of very specific regions through a spot fixing technique. Anumber of ocular pathologies, such as drusen or holes, tend to becircularly symmetrical. The ability to select a spot on an image,position a set of crosshairs centered at the spot, and vary the size anddensity of the crosshairs can increase the efficiency of editing suchcircularly symmetrical spots.

FIG. 5 illustrates a spot-based workflow in accordance with anembodiment. In step (i) of the workflow of FIG. 5 , the softwaregenerates a new segmentation based on a user's delineations ofpre-defined slices, and the user places a set of crosshairs or amulti-armed star on a region specified as needing editing. Thecrosshairs or multi-arm star can have any size or density, where thedensity of the shape is defined as the number of arms of the shape. Forexample, a plus sign is a shape with four arms as shown in FIG. 5 . Apath is then defined as in step (ii) of FIG. 5 as the walking of theboundary of the crosshairs. A slice corresponding to this walking pathis generated through the volume of the sample, and the currentdelineation associated with this slice is displayed in step (iii) ofFIG. 5 . The user is then prompted to either edit this current layer orgenerate a new one. When the user is finished editing the layerdelineation, a new segmentation is generated in step (iv) of FIG. 5 .The user can iterate this process over the same or other zones untilsatisfied with the full segmentation.

In some embodiments, the generation of layer data is throughre-segmentation using the user's delineations as input data. There-segmentation can be accomplished by interpolation. Interpolationsperformed as part of the workflows can be carried out using, forinstance, a scattered point interpolation algorithm. According to anembodiment of the present invention, the scattered point interpolationalgorithm can be used to generate a set of interpolated layersegmentation values based on a set of pre-determined or user-definedlayer delineations. The interpolations can be carried out using, forinstance, a spline interpolation algorithm. According to an embodimentof the present invention, the spline interpolation algorithm can be usedto generate piecewise polynomials between pre-determined or user-definedlayer delineation values. According to an embodiment of the presentinvention, the interpolation could also be carried out via Shephardinterpolation, Natural Neighbor interpolation, Wiener interpolation,Poisson interpolation or radial basis function interpolation. One ofordinary skill in the art would appreciate that there are numerous typesof algorithms that could be used for interpolations, and embodiments ofthe present invention are contemplated for use with any such algorithmfor interpolation.

In addition to or substitution for interpolation, other ways to useoperator input delineations to generate a full surface would be toresegment a layer using the delineations as a priori knowledge, forinstance by: (i) building a classifier using the delineations astraining data and applying that classifier to the non-delineated areas;or (ii) initializing an active surface model at the site of the userdelineations and iterating the surface to optimally segment the layerbased on the volume data. One of ordinary skill in the art wouldappreciate that there are numerous methods for resegmenting layers usingdelineations as a priori knowledge, and embodiments of the presentinvention are contemplated for use with any appropriate method for suchresegmentation.

Turning now to FIG. 6A, an exemplary method of determining biologicallayer structure in a three-dimensional (3D) tomography is shown, inaccordance with an embodiment of the present invention. The processstarts with the system being engaged at step 101. At step 102, thesystem first renders a first two-dimensional (2D) image produced from a3D tomographic model. The 3D tomographic model is generally produced viaone or more computing devices configured to analyze and processgraphical data. In this example, the 3D tomographic model represents asubject (e.g., retina). Being 3D, the tomographic model has 3 axes(e.g., x-axis, a y-axis, a z-axis). Further, in this example, the first2D image represents a first cross section of the subject. This firstcross section is the intersection of the subject and a first surface.The first surface in this example being parallel to the z-axis.

At step 103, the system generates and provides a graphicalrepresentation of the first 2D image to an operator. Generally, thegraphical representation will be presented to an operator via a displayelement attached or communicatively connected to a computing deviceoperated by the operator.

At step 104, the system receives input from the operator. Input isgenerally received by one or more input devices (e.g., keyboard, mouse,capacitive touch screen) attached to or communicatively connected to thecomputing device operated by the operator. The received input from theoperator is a first delineation of one or more layers within thedisplayed first 2D image. It should be noted that the computing deviceof the operator may be separate from the computing device processinggraphical and data elements associated with the system. In certainembodiments they may be the same computing device, while in others, twoor more computing devices can be communicatively connected (e.g., via acommunications means utilizing one or more data transmission means, suchas wireless networking or wired networking) to provide data back andforth between the computing devices. One of ordinary skill in the artwould appreciate that there are numerous configurations that could beutilized with embodiments of the present invention, and embodiments ofthe present invention are contemplated for use with any appropriateconfiguration of computing devices.

At step 105, the system relates the first delineation received from theoperator input with a first layer of the one or more layers. Relation ofthe first delineation received from the operator input may be achievedby, for instance, identifying and matching points provided by theoperator identifying the first delineation with the first layer.

At step 106, the system renders a second 2D image produced from the 3Dtomographic model. In this example, the second 2D image represents asecond cross section of the subject, representing an intersection of thesubject and a second surface which is either (i) a nonplanar surfaceparallel to the z-axis, or (ii) a planar surface parallel to the z-axisand not parallel to any plane comprising (a) the z-axis and (b) eitherthe x-axis or the y-axis.

At step 107, the system generates and provides a graphicalrepresentation of the second 2D image to the operator for further input.

At step 108, the system receives input from the operator. This input isrelating to a second delineation of one or more layers within thedisplayed second 2D image.

At step 109, the system associates the second delineation with the firstlayer. Association of the second delineation with the first layer may beachieved by, for instance, identifying and matching points provided bythe operator identifying the second delineation with the first layer.

At step 110, the system generates layer data for the 3D tomographicmodel by interpolating the first and second delineations and finallyoutputs layer data for the 3D tomographic model at step 111. In certainembodiments, the interpolation could be performed by a procedure thatuses the a priori information from the first and second delineations togenerate the unknown layer data. One of ordinary skill in the art wouldappreciate that there are numerous procedures that use a prioriinformation to generate unknown layer data, and embodiments of thepresent invention are contemplated for use with any appropriateprocedure.

Turning now to FIG. 6B, an exemplary method of determining biologicallayer structure in a three-dimensional (3D) tomography is shown, inaccordance with an embodiment of the present invention. The processstarts at step 201, which is a continuation of the exemplary methodshown in FIG. 6A. At step 202, the system renders a third 2D imageproduced from the 3D tomographic model. In this example, the third 2Dimage represents a third cross section of the subject. The system mayalso be further configured to compute a third delineation of one or morelayers within the displayed third 2D image using the computer.

At step 203, the system displays the third 2D image and the thirddelineation of layers to the operator. The system then waits for andreceives operator input associated with the editing of the thirddelineation of layers (step 204). After receiving the operator input,the system generates an edited third delineation of layers.

At step 205, the system associates the edited third delineation with thefirst layer.

At step 206, the system generates layer data for the 3D tomographicmodel by interpolating the first and second delineations and the editedthird delineation. As detailed above, interpolation can be carried outby the system using any number of interpolation algorithms, such as ascattered point interpolation algorithm or a spline interpolationalgorithm or could be accomplished through a procedure that uses the apriori information from the first and second delineations to generatethe unknown layer data.

At step 207, the system determines a series of confidence values,wherein each confidence value is assigned to a predetermined point inthe x-y plane of the subject. The confidence values may be determined ina manner detailed previously herein, such as minimizing the maximum,mean, or total distance from all “low confidence” locations.

At step 208, the system defines a second path in the x-y plane of thesubject. In this example, the second path is selected to comprise pointsin the x-y plane having a mean or minimum confidence value below atarget value. At this point, the process terminates at step 209.

Turning now to FIG. 6C, an exemplary method of determining biologicallayer structure in a three-dimensional (3D) tomography is shown, inaccordance with an embodiment of the present invention. The processstarts at step 201, which is a continuation of the exemplary methodshown in FIG. 6A. At step 302, the system receives operator inputdefining a second path in the x-y plane of the subject.

At step 303, the system renders a third 2D image produced from the 3Dtomographic model using the computer. In this example, the third 2Dimage represents a third cross section of the subject. The third crosssection is the intersection of the subject and a third surface, and thethird surface consists of each line parallel to the z-axis of thesubject and passing through the second path in the x-y plane of thesubject.

At step 304, the system generates and provides display data forprovision to the operator, wherein the display presented to the operatoris the third 2D image.

At step 305, the system receives operator input of a third delineationof one or more layers within the displayed third 2D image. The systemthen associates the third delineation with the first layer.

Finally, at step 306, the system generates layer data for the 3Dtomographic model by interpolating the first, second and thirddelineations using the computer. As detailed above, interpolation can becarried out by the system using any number of interpolation algorithms,such as a scattered point interpolation algorithm or a splineinterpolation algorithm. As detailed above, interpolation can be carriedout by the system using any number of interpolation algorithms, such asa scattered point interpolation algorithm or a spline interpolationalgorithm or could be accomplished through a procedure that uses the apriori information from the first and second delineations to generatethe unknown layer data. The process then terminates at step 307.

According to an embodiment of the present invention, the system andmethod may be configured to process and communicate data to and may beused in conjunction or through the use of one or more computing devices.As shown in FIG. 7 , One of ordinary skill in the art would appreciatethat a computing device 400 appropriate for use with embodiments of thepresent application may generally be comprised of one or more of aCentral processing Unit (CPU) 401, Random Access Memory (RAM) 402, astorage medium (e.g., hard disk drive, solid state drive, flash memory,cloud storage) 403, an operating system (OS) 404, one or moreapplication software 405, one or more display elements 406, one or moreinput/output devices/means 407 and one or more databases 408. Examplesof computing devices usable with embodiments of the present inventioninclude, but are not limited to, personal computers, smartphones,laptops, mobile computing devices, tablet PCs and servers. Certaincomputing devices configured for use with the system do not need all thecomponents described in FIG. 7 . For instance, a server may notnecessarily include a display element. The term computing device mayalso describe two or more computing devices communicatively linked in amanner as to distribute and share one or more resources, such asclustered computing devices and server banks/farms. One of ordinaryskill in the art would understand that any number of computing devicescould be used, and embodiments of the present invention are contemplatedfor use with any computing device.

Turning to FIG. 8 , according to an embodiment of the present invention,a system for generation or editing of layer delineations is comprised ofone or more communications means 501, one or more data stores 502, aprocessor 503, memory 504, an image processing module 505 and layerdelineation processing module 506. FIG. 9 shows an alternativeembodiment of the present invention, comprised of one or morecommunications means 601, one or more data stores 602, a processor 603,memory 604, an image processing module 605, a layer delineationprocessing module 606 and an image capture module 607. The variousmodules described herein provide functionality to the system, but thefeatures described and functionality provided may be distributed in anynumber of modules, depending on various implementation strategies. Oneof ordinary skill in the art would appreciate that the system may beoperable with any number of modules, depending on implementation, andembodiments of the present invention are contemplated for use with anysuch division or combination of modules as required by any particularimplementation. In alternate embodiments, the system may have additionalor fewer components. One of ordinary skill in the art would appreciatethat the system may be operable with a number of optional components,and embodiments of the present invention are contemplated for use withany such optional component.

According to an embodiment of the present invention, the communicationsmeans of the system may be, for instance, any means for communicatingdata over one or more networks or to one or more peripheral devicesattached to the system. Appropriate communications means may include,but are not limited to, wireless connections, wired connections,cellular connections, data port connections, Bluetooth® connections, orany combination thereof. One of ordinary skill in the art wouldappreciate that there are numerous communications means that may beutilized with embodiments of the present invention, and embodiments ofthe present invention are contemplated for use with any communicationsmeans.

According to an embodiment of the present invention, the layerdelineation processing module is configured to process the variousdelineations between layers as detailed herein. In a preferredembodiment, the layer delineation processing module is configured toprovide a fully automated segmentation algorithm (e.g., retinal layersegmentation). In other embodiments, the layer delineation processingmodule is configured to work in conjunction with operator provided inputto provide a semi-automated segmentation process.

According to an embodiment of the present invention, the imageprocessing module works in conjunction with the layer delineationprocessing module and is configured to render 2D and 3D images andgenerate display data for consumption by the operator. Generation,processing and rendering means and methods utilized by the imageprocessing module are detailed previously herein. Further, one ofordinary skill in the art would appreciate that there are numerousmethods for processing 2D and 3D images and generating display data, andembodiments of the present invention are contemplated for use with anyappropriate means for processing and generating such data.

According to an embodiment of the present invention, the Image capturemodule is configured to capture image data to be used in generation ofthe 2D and 3D images used by other components of the system. Forinstance, the Image capture module could be configured to receive inputfrom a imaging device (e.g., 2D or 3D imaging device, CT scanner, MMscanner, optical coherence tomography scanner) and process such inputinto a subject for use in the methods detailed herein.

Throughout this disclosure and elsewhere, block diagrams and flowchartillustrations depict methods, apparatuses (i.e., systems), and computerprogram products. Each element of the block diagrams and flowchartillustrations, as well as each respective combination of elements in theblock diagrams and flowchart illustrations, illustrates a function ofthe methods, apparatuses, and computer program products. Any and allsuch functions (“depicted functions”) can be implemented by computerprogram instructions; by special-purpose, hardware-based computersystems; by combinations of special purpose hardware and computerinstructions; by combinations of general purpose hardware and computerinstructions; and so on—any and all of which may be generally referredto herein as a “circuit,” “module,” or “system.”

While the foregoing drawings and description set forth functionalaspects of the disclosed systems, no particular arrangement of softwarefor implementing these functional aspects should be inferred from thesedescriptions unless explicitly stated or otherwise clear from thecontext.

Each element in flowchart illustrations may depict a step, or group ofsteps, of a computer-implemented method. Further, each step may containone or more sub-steps. For the purpose of illustration, these steps (aswell as any and all other steps identified and described above) arepresented in order. It will be understood that an embodiment can containan alternate order of the steps adapted to a particular application of atechnique disclosed herein. All such variations and modifications areintended to fall within the scope of this disclosure. The depiction anddescription of steps in any particular order is not intended to excludeembodiments having the steps in a different order, unless required by aparticular application, explicitly stated, or otherwise clear from thecontext.

Traditionally, a computer program consists of a finite sequence ofcomputational instructions or program instructions. It will beappreciated that a programmable apparatus (i.e., computing device) canreceive such a computer program and, by processing the computationalinstructions thereof, produce a further technical effect.

A programmable apparatus includes one or more microprocessors,microcontrollers, embedded microcontrollers, programmable digital signalprocessors, programmable devices, programmable gate arrays, programmablearray logic, memory devices, application specific integrated circuits,or the like, which can be suitably employed or configured to processcomputer program instructions, execute computer logic, store computerdata, and so on.

It will be understood that a computer can include a computer-readablestorage medium and that this medium may be internal or external,removable and replaceable, or fixed. It will also be understood that acomputer can include a Basic Input/Output System (BIOS), firmware, anoperating system, a database, or the like that can include, interfacewith, or support the software and hardware described herein.

Embodiments of the system as described herein are not limited toapplications involving conventional computer programs or programmableapparatuses that run them. It is contemplated, for example, thatembodiments of the invention as claimed herein could include an opticalcomputer, quantum computer, analog computer, or the like.

Regardless of the type of computer program or computer involved, acomputer program can be loaded onto a computer to produce a particularmachine that can perform any and all of the depicted functions. Thisparticular machine provides a means for carrying out any and all of thedepicted functions.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Computer program instructions can be stored in a computer-readablememory capable of directing a computer or other programmable dataprocessing apparatus to function in a particular manner. Theinstructions stored in the computer-readable memory constitute anarticle of manufacture including computer-readable instructions forimplementing any and all of the depicted functions.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

The elements depicted in flowchart illustrations and block diagramsthroughout the figures imply logical boundaries between the elements.However, according to software or hardware engineering practices, thedepicted elements and the functions thereof may be implemented as partsof a monolithic software structure, as standalone software modules, oras modules that employ external routines, code, services, and so forth,or any combination of these. All such implementations are within thescope of the present disclosure.

In view of the foregoing, it will now be appreciated that elements ofthe block diagrams and flowchart illustrations support combinations ofmeans for performing the specified functions, combinations of steps forperforming the specified functions, program instruction means forperforming the specified functions, and so on.

It will be appreciated that computer program instructions may includecomputer executable code. A variety of languages for expressing computerprogram instructions are possible, including without limitation C, C++,Java, JavaScript, Python, assembly language, Lisp, and so on. Suchlanguages may include assembly languages, hardware descriptionlanguages, database programming languages, functional programminglanguages, imperative programming languages, and so on. In someembodiments, computer program instructions can be stored, compiled, orinterpreted to run on a computer, a programmable data processingapparatus, a heterogeneous combination of processors or processorarchitectures, and so on.

In some embodiments, a computer enables execution of computer programinstructions including multiple programs or threads. The multipleprograms or threads may be processed more or less simultaneously toenhance utilization of the processor and to facilitate substantiallysimultaneous functions. By way of implementation, any and all methods,program codes, program instructions, and the like described herein maybe implemented in one or more thread. The thread can spawn otherthreads, which can themselves have assigned priorities associated withthem. In some embodiments, a computer can process these threads based onpriority or any other order based on instructions provided in theprogram code.

Unless explicitly stated or otherwise clear from the context, the verbs“execute” and “process” are used interchangeably to indicate execute,process, interpret, compile, assemble, link, load, any and allcombinations of the foregoing, or the like. Therefore, embodiments thatexecute or process computer program instructions, computer-executablecode, or the like can suitably act upon the instructions or code in anyand all of the ways just described.

The functions and operations presented herein are not inherently relatedto any particular computer or other apparatus. Various general-purposesystems may also be used with programs in accordance with the teachingsherein, or it may prove convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will be apparent to those of skill in theart, along with equivalent variations. In addition, embodiments of theinvention are not described with reference to any particular programminglanguage. It is appreciated that a variety of programming languages maybe used to implement the present teachings as described herein, and anyreferences to specific languages are provided for disclosure ofenablement and best mode of embodiments of the invention. Embodiments ofthe invention are well suited to a wide variety of computer networksystems over numerous topologies. Within this field, the configurationand management of large networks include storage devices and computersthat are communicatively coupled to dissimilar computers and storagedevices over a network, such as the Internet.

The functions, systems and methods herein described could be utilizedand presented in a multitude of languages. Individual systems may bepresented in one or more languages and the language may be changed withease at any point in the process or methods described above. One ofordinary skill in the art would appreciate that there are numerouslanguages the system could be provided in, and embodiments of thepresent invention are contemplated for use with any language.

While multiple embodiments are disclosed, still other embodiments of thepresent invention will become apparent to those skilled in the art fromthis detailed description. The invention is capable of myriadmodifications in various obvious aspects, all without departing from thespirit and scope of the present invention. Accordingly, the drawings anddescriptions are to be regarded as illustrative in nature and notrestrictive.

What is claimed is:
 1. A method for analyzing a three-dimensionaltomographic image, the method comprising: presenting a plurality oftwo-dimensional slices from a three-dimensional tomographic image,wherein at least one of the two-dimensional slices comprises across-section that is non-planar or is planar and not axially aligned;editing one or more of the plurality of two-dimensional slices based onan input from a user; and propagating information from the one or moreedited two-dimensional slices to generate an edited three-dimensionalmodel wherein propagating comprises interpolating the edited one or moretwo-dimensional slices using an interpolation algorithm.
 2. The methodof claim 1, wherein the three-dimensional tomographic image has anx-axis, a y-axis, and a z-axis, and wherein at least one of theplurality of two-dimensional slices corresponds to a plane that isparallel to the z-axis and not parallel to either the y-axis or thex-axis.
 3. The method of claim 1, wherein at least one of the pluralityof two-dimensional slices corresponds to a curve drawn in an x-y planeof the three-dimensional tomographic image.
 4. The method of claim 3,wherein the curve is drawn by the user.
 5. The method of claim 3,wherein the curve is generated by an algorithm to cover a region with ahigh likelihood of error.
 6. The method of claim 1, wherein the curve isa circle.
 7. The method of claim 1, wherein the two-dimensional slicesindicate one or more structural boundaries in the three-dimensionaltomographic image.
 8. The method of claim 7, wherein one or morestructural boundaries comprise one or more layer delineations.
 9. Themethod of claim 8, wherein editing comprises adjusting a layerdelineation in the two-dimensional slice.
 10. The method of claim 8,wherein editing comprises adding or removing a layer delineation in thetwo-dimensional slice.
 11. The method of claim 8, wherein the pluralityof two-dimensional slices are predetermined by minimizing a distancefrom a region determined to have layer delineations that are likely tobe incorrect.
 12. The method of claim 8, wherein the one or more layerdelineations correspond to retinal layer segmentation.
 13. The method ofclaim 1, further comprising prompting the user to edit each of theplurality of two-dimensional slices.
 14. The method of claim 1, whereinthe plurality of two-dimensional slices are presented to the user on ascreen.
 15. The method of claim 1, wherein the plurality oftwo-dimensional slices are presented to the user one after another. 16.The method of claim 1, wherein at least two of the plurality oftwo-dimensional slices intersect with each other.
 17. The method ofclaim 1, wherein an edit made to a first two-dimensional slice isapplied to a second two-dimensional slice.
 18. The method of claim 1,wherein propagating information comprises applying an algorithm selectedfrom: a scattered point interpolation algorithm; a spline interpolationalgorithm; Shephard interpolation; natural neighbor interpolation;Wiener interpolation; Poisson interpolation; and radial basis functioninterpolation.
 19. The method of claim 1, wherein the three-dimensionaltomographic image is an optical coherence tomography image.
 20. Themethod of claim 1, wherein the three-dimensional tomographic image is ofa retina.